eTRIMS Scene Interpretation Datasets

We describe two image datasets for learning and evaluating interpretations of man-made scenes. The datasets consist of (A) 110 and (B) 200 fully annotated images from the building façade domain. Dataset A emphasises prominent object structures, and has been manually double-checked. Dataset B puts more emphasis on single object recognition on a larger set of classes, and has not been double checked. For both datasets, we define an object partonomy as a set of related labelled polygons, which are provided in an XML format based on the MIT LabelMe database. The datasets can be used as ground truth for training and evaluating object detection and classification algorithms, structure detection and classification algorithms as well as for evaluating complete interpretations of structured scenes, which makes use of the partonomy and taxonomy of objects.